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[57] ABSTRACT 

A virtual reality flight control system displays to the 
pilot the image of a scene surrounding a vehicle or pod 
having six degrees of freedom of acceleration or veloc- 
ity control by the pilot and traveling through inertial 
space, the image itself including a superimposed figure 
providing the pilot an instant reference of orientation 
consisting of superimposed sets of geometric figures 
whose relative orientations provide the pilot an instan- 
taneous feel or sense of orientation changes with respect 
to some fixed coordinate system. They include a first set 
of geometric figures whose orientations are fixed to the 
pilot's vehicle and a second set of geometric figures 
whose orientations are fixed with respect to a fixed or 
interstellar coordinate system. The first set of figures is 
a first set of orthogonal great circles about the three 
orthogonal axes of the flight vehicle or pod and cen- 
tered at and surrounding the pilot's head, while the 
second set of figures is a second set of orthogonal great 
circles about the three orthogonal axes of a fixed or 
interstellar coordinate system, also centered at and sur- 
rounding the pilot's head. 

17 Claims, 6 Drawing Sheets 
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by the trainee oriented in accordance with the pilot's 
VIRTUAL REALITY FLIGHT CONTROL DISPLAY head, as disclosed in U.S. Pat No. 4,303,394. 

WITH SIX-DEGREE-OF-FREEDOM Because of the relatively limited movements and 

CONTROLLER AND SPHERICAL ORIENTATION rotations simulated by a flight simulator, it is possible to 
OVERLAY 5 actually move the simulator to simulate many of the 

accelerations commanded by the trainee, as disclosed in 
BACKGROUND OF THE INVENTION U.S. Pat No. 4,487,410. Since most accelerations corre- 

I. Origin of the Invention s P° nd t0 Perturbations from horizontal flight, simulat- 

The invention described herein was made in the per- " m £ su ? h accelerations is quite practical. This is compa- 
formance of work under a NASA contract, and is sub- 10 ™* le m ^entertainment field to the Disney-Lucas 
ject to the provisions of Public Law 96-517 (35 USC Ster-Tours nde at Disneyland. 
202) in which the Contractor has elected not to retain . A ™°^y of the foregoing is that known flight 
t - t jg simulators are not particularly useful for simulating 

2 Technical Field flight beyond earth's atmosphere, and particularly flight 

The inventioD is related to virtual reality systems and 15 f^u^ inertial space w^e ttoe is no preferred ^or- 
in particular to the type of virtual reaUty systems pro- ^ ^««tion or homon. Smce there « no partou- 
j f «• • ... , : . . . . lar earth or planetary coordinate system in flight 

posed for use m flight simulation and pilot training ^ , . K / , , , - J ,„ . A, 
j . „ w II *i. * ^ ■ n- through mertial space by which to define horizontal , 

deuces. More specifically, the invention i is a flight con- such ^ ^ ^ per turbations from 

trol system for virtual navigation m inerUal space hav- 2Q a fe ^ ed « horizontar orientation Tins is because an 
mg no ground plane or preferred orientation. J^e flfght m sp ^ ^ ^ „ down „ 

wf a i °n v ^ ^ i -.u or straight "up" or in an "upside down" orientation 

Virtue reality displays have become practical with with^y stations though 3^ degrees, any of which 

the combination of computers and video image proccs- wou]d te ^ ^ the * bilities of % m ht simuktor 

sors. ^mputer-controUed video imaging has led to 2$ Morcover , jf a standard & ht simulator ^ modified 

computerized simulation of images, which in turn has for ^ ^ bating fli ht mertial b 

led to simulation of images resrx>nsive to user-con- removing ^ p 4 e or ho £ on from £ e simu . 

trolled interactions. In particular v Foley, "Interfeces for lated ^ ^ trainee wou i dhave no reference in the 

tt™ 0 "* Ame ™ an > 0ctob f ' image from which he could reliably discern his orienta- 
1987, pages 127-134, discloses, for example, the use of a 30 tion ^ rotatioils . 

wired glove worn by the computer user and connected This is in fact a fundamental problem in any flight 

to a computer that generates images on a monitor re- through in Cr tial space, simulated or real: giving the pilot 

sponsive to ^movements ^of the glove to give the user the a ^ ^ accuratc rcfcrcnce t0 sensc orientation, 

sensation of handling objects, a virtual reality. Fisher et However, there is no technique known that addresses 

al, "Virtual Environment Display System," ACM 1986 35 ^ pro blem. For example, U.S. Pat. No. 4,984,179 

Workshop on Interactive 3D Displays, Oct. 23-24, 1986, discloses a computer-generated three-dimensional dis- 

Chapel Hill, N.C., discloses head-mounted stereoscopic play th at permits the viewer to command rotations 

displays controlled by the users movements, such as about any axis and to view the results m this display, but 

might be useful in practicing movements required to ^ not capa bl e of simulating translation^ movement 

perform certain tasks. Such virtual reality systems, 40 a i ong more than one axis. Moreover, since this tech- 

which typically require supercomputers, were fore- ^q^c implicitly assumes a ground plane or horizon, no 

shadowed by flight simulators for pilot training, which provision is made for solving the problem of maintain- 

used ordinary computers to control the image display. m g m the displayed image a reference of the trainee's 

Head-mounted, position sensitive displays were shown orientation in three dimensions, 

by D. Engelbart as early as 1962. 45 Suppose a person, the traveller, is immersed in a vir- 

Flight simulators of the type employed in aircraft tual reality. The virtual reality is made of graphic im- 
pilot training provide a frame of reference in the image ages synthesized by computer from real or model data 
presented to a trainee, dominated by a ground plane, and projected into the traveller's eyes by means of bin- 
giving the pilot a constant horizontal reference. The ocular, full field-of-view display screens, such as the 
flight simulator alters the scene observed by the trainee 50 EyePhones TM of VPL, Inc. Let us call these, generi- 
in accordance with flight control commands from a joy cally, virtual reality displays (VRDs). Naturally, the 
stick controlled by the trainee. This joystick serves the images may be accompanied by synthesized audio and 
same function in flight simulation as the glove serves in other media to further enhance the illusion of reality, 
the virtual reality system of the publication by Foley but the subject here is the images. The projection pa- 
referenced above. The image processing is limited in 55 rameters that determine the apparent parallax, point of 
that the flight movements are treatable as perturbations view, and gazing direction of the traveller are deter- 
from horizontal flight. mined by the virtual reality control system (VRCS). 

In one type disclosed in U.S. Pat No. 5,137,450, a Most proposed or existing VRCS's involve devices that 

number of video projectors project different perspec- sense the position and orientation of the traveller's head 

tive views of a scene to different "windows" surround- 60 and adjust the projection parameters to maintain the 

ing the trainee's field of view. In another type disclosed illusion that the traveller is immersed in a real scene, 

in U.S. Pat. Nos. 5,134,521; 4,985,762; 4,310,849 and When the traveller turns his head, a new gazing direc- 

4,348, 1 85, the display is a device very close to the train- tion is calculated from the new head orientation sensed 

ee's eyes, facilitating a more realistic panoramic image. by the VRCS. Usually, the computed gazing direction 

In such displays, the image follows the movements of 65 is the same that would obtain in a real reality. This 

the trainee's head to present a more realistic image. maximally real gazing direction most maintains the 

Flight simulators employing computer-generated illusion of reality. Likewise, when the traveller moves 

images have presented images of targets to be viewed around within the sensing range of the VRCS, a new 
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point of view is calculated from the sensed position. instantaneous reference of orientation with respect to 

Many interesting mappings from the position and orien- some fixed coordinate system. 

tation sensed by the VRCS to the projection parameters It is a related object of the invention to provide such 

can be used to create a variety of illusions, and such a set of geometric figures superimposed on the scene 

mappings are a fertile ground for experimentation and 5 and apparently surrounding the pilot that include a first 

invention. The subject, here, however, is the sensing of set of geometric figures whose orientations are fixed to 

the position and orientation* Let us assume the most the pilot's vehicle and a second set of geometric figures 

straightforward mappings for the time being. The whose orientations are fixed with respect to a fixed 

VRCS must repeatedly sense position and orientation, - global, interplanetary or interstellar coordinate system, 

and the rest of the virtual reality system must quickly 10 it is a yet further object of the invention to provide, 

calculate a new point of view and gazing direction. This ^ suc h a virtual reality display, the superimposed set of 

must be done quickly enough that the viewer cannot geometric figures which include a first set of orthogonal 

perceive any delay between the instant of movement to great circles about the three orthogonal axes of the 

a new position and orientation and the instant of display flight vehicle or pod and centered at and surrounding 

of a new projection using the new point of view and 15 ^ pilot » s head ^ a second set of orthogonal great 

gazing direction, that is, within the persistence thresh- circles about ^ ort hogonal axes of a fixed or 

old of the human visual system. Typically, so long as interstellar coordinate system, also centered at and sur- 

the delay is less than about 1/30 seconds, the illusion of r0U nding the pilot's head, 

reality is maintauied. The head-sensing control meta- it is a still further object of the invention to provide, 

phor is adequate for some two-dimensional virtual reali- 20 ^ such a ^ ^ tick-markings or off-axis secondary 

ties say walking around a virtual office or moving ^ m , hg ^ e ach of the main 

withui a virtual tenms court, but is not adequate, alone, ^ ^ ^ Uot ^ ^ 

for three-dimensional realities How is one to fly a w- ^ of rotation about ^ 

^auTrfaneJor e^^ is Qbject of ^ t£> a 

the case of airplanes providing the traveller with a 25 * virtual J reality ^ lay t0 scan 

io y stick, rudder pedals, and a throttle is probably the . , * . . 

bit thkg to do. Travellers who know how to fly* real a .f of veloc,ties for °*^ oa » 

airplanes will find these controls immediately useful in a lg * 

virtual spaces, and travellers who do not know how to SUMMARY OF THE DISCLOSURE 
fly will learn something useful for the real world by 30 _ . ■ . . . n . - A . 
using these controls. One might, on option, "fudge" the ™ e ™«*» » * virtual control system 
exact metaphor of flight to provide quick, non-physical ? at dls P la y? t0 P dot W of a scene surround- 
position changes and play other useful games with the m * a vel J? cle or ^mg six degrees of free- 
kinematics. However, the airplane metaphor presup- dom ° f acceleration or velocity control by the pilot and 
poses a horizon or ground plane, so is not fully three-di- 35 traveling through inertial space, the image itself includ- 
mensional. Some other control metaphor is required for m S a superimposed figure providing the pilot an instant 
untethered, free flight in three-dimensional space. reference of orientation. 

Accordingly, it is an object of the invention to pro- 1x1 order t0 P^vide such instant reference of onenta- 
vide an apparatus for displaying to a pilot or trainee the tion t0 ^ P aot * ^ includes a superimposed set 
image of a scene surrounding a vehicle traveling in 40 of geometric figures whose relative orientations pro- 
inertial space in response to rotational and translational vide the pilot an instantaneous feel or sense of orienta- 
acceleration commands by the pilot in six degrees of tion changes with respect to some fixed coordinate 
freedom. ^em- 
it is a further object of the invention to provide a Preferably, such a set of geometric figures is superim- 
flight control apparatus having six degrees of freedom 45 posed on the scene observed from the pod or vehicle 
of acceleration or velocity control by a pilot that dis- apparently surrounds the pilot They include a first 
plays to the pilot the image of a scene surrounding a set of geometric figures whose orientations are fixed to 
vehicle or pod containing the pilot traveling through &e pilot's vehicle and a second set of geometric figures 
inertial space, the image itself including a superimposed whose orientations are fixed with respect to a global, 
figure providing the pilot an instant reference of orien- 50 interplanetary or interstellar coordinate system. The 
tation. frst set °f figures is a first set of orthogonal great circles 
It is a related object of the invention to provide a about the three orthogonal axes of the flight vehicle or 
flight control apparatus having six degrees of freedom pod and centered at and surrounding the pilot's head, 
of acceleration or velocity control by a pilot that dis- while the second set of figures is a second set of orthog- 
plays to the pilot the image of a scene surrounding a 55 onal great circles about the three orthogonal axes of a 
vehicle or pod containing the pilot traveling through fixed or interstellar coordinate system, also centered at 
inertial space, the image itself including a superimposed and surrounding the pilot's head. There are tick mark- 
set of geometric figures whose relative orientations ings (or off-axis circles) at periodic arc lengths along 
provide the pilot an instantaneous reference of orienta- each of the great circles that permit the pilot to instantly 
tion with respect to some fixed coordinate system. 60 gauge approximate angles of rotation about any axis, 

It is a still further object of the invention to provide a An alternative feature of the invention is a velocity 
virtual reality display corresponding to a flight control scan option that permits the pilot in such a virtual real- 
apparatus having six degrees of freedom of acceleration ity display to instantly scan among a set of velocities for 
or velocity control by a pilot that displays to the pilot easy navigation in simulated flight. 

^L°^^r^^ T ^ 65 BRIEF DESCRIPTION OF THE DRAWINGS 

image itself including a superimposed set of geometric FIG. 1 illustrates a virtual traveller in a virtual reality 
figures whose relative orientations provide the pilot an system including a head-worn display and a six-degree- 
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of-freedom control ball employed in carrying out the Nature that human beings (and other animals) feel more 
invention. oriented and less "lost" when a ground plane is visible, 

FIG. 2 illustrates a preferred six-degree-of-freedom and the space pod supports control in the presence of 
control ball employed in the system of FIG. 1. one. The distinguishing feature of the space pod is that 

FIG. 3 illustrates the traveller's perception of the 5 a ground plane is neither required nor inherent in the 
virtual reality display of a spherical pod on which are system. Let us use the right-hand control, 
superimposed two sets of orthogonal great circles sur- The first problem is to go to the observed star. Sup- 
rounding the traveller and corresponding to the three pose we see the red star about sixty degrees above us 
orthogonal axes of the pod and of the three orthogonal and about forty-five degrees to the left. We can move 
axes of a fixed or interstellar coordinate system. 10 our head and see it. That is where we want to go. We 

FIG. 4 illustrates the two sets of great circles of FIG. strike button A and space navigation circles are over- 
3 separately. layed on the field of stars. The navigation circles are a 

FIG. 5 illustrates how the two sets of great circles set of three blue and three yellow circles surrounding 
may appear in one view of the traveller. the traveller. The circles have major tick marks every 

FIG. 6 is a block diagram illustrating the architecture IS thirty degrees, making twelve in all, and minor tick 
of the virtual reality flight control system of the inven- marks every degree. Alternatively, the tick marks may 
tion. be replace by off-axis circles. The number of marks or 

FIG. 7 is a block flow diagram illustrating the opera- off-axis circles per arc degree is programmable or con- 
tion of the system of FIG, 6. trolled by the pilot. The blue circles are fixed to the 

20 global interstellar coordinate system and the yellow 
circles are fixed to the local space pod coordinate sys- 
tem. These circles allow the traveller to visualize pod 
Referring to FIG. 1, a traveler through inertial space orientation quantitatively, 
such as a pilot (in a real space craft) or a trainee (in a We twist the ball to the left (like the dial of a floor 
room) 10 sits in a chair 12 with a right arm rest 14 on 25 safe combination lock) and back (like the throttle grip of 
which a six-degree-of-freedom control ball 16 (FIG. 2) a motorcycle). The yellow navigation circles start to 
is mounted for easy control by the pilot 10. The pilot 10 slide within the blue ones and the red star starts to move 
wears a pilot head rotation sensor 17 and a set binocular toward front and center. Simultaneously, we push the 
full-field-of-view display screens 18. The control ball 16 ball slightly forward and pull it up a little. That action 
generates command signals transmitted to a computer 30 starts us moving toward the destination star, and we 
19. The computer 19 produces images which change know we are moving because the other stars in the 
with movement of the pilot's head (to provide a realistic peripheral field of view start to slide past slowly. As the 
"surround" image). The computer 19 also changes the destination star moves more to front and center, we 
images with movement of the pilot's virtual "craft" or slowly relax the twists on the ball and the springs in the 
pod in response to the command signals from the con- 35 stalk restore it to its resting position. Simultaneously we 
trol ball 16. relax the upward pull on the ball. The star is now di- 

As shown in FIG. 2, the control ball 16 is translatable rectly ahead, and we are pushing the ball forward. The 
in two directions along each of three orthogonal axes, longer we push, the faster we go. 
labelled X, Y, Z in the drawing, and also rotatable about When we have achieved a controllable speed, we 
each one of those axes, the control ball 16 permitting 40 relax completely on the ball, and we continue gliding 
any and all combinations of such translations and rota- (according to Newton's First Law) toward the star. We 
tions to be made simultaneously as the desired. twist the ball slightly to the right (like a doorknob) to 

Let us seat the traveller in a large, comfortable arm align the horizontal yellow (pod) navigation circle as 
chair. The traveller is wearing virtual reality displays. closely as possible with the most horizontal-looking 
Let us put the control ball 16, which may be the size of 45 blue (space) circle. We see the stars rushing past our 
a softball, in a hand of the user. The ball 16 is secured to peripheral view correspondingly twist helically to the 
an arm of the chair by an elastic stalk. The stalk is in- left. We have now achieved the desired orientation. We 
strumented to pick up displacements and twists in three strike button A again and the navigation circles disap- 
directions. Hence, each ball furnishes six degrees of pear. The star is now apparently growing in size as we 
freedom of input to the VRCS. Balls like this are com- 50 approach. We begin to see several planets orbiting the 
mercially available; the SpaceBall TM is one such. Let star. We are about as close as we want to be. We have 
us also put three buttons, say A, B, and C on the outside two options, now. First, we may pull back hard on the 
of the arm of the chair so that the traveller can easily control ball, applying maximum braking and eventually 
strike them simply by removing his hand from the ball coming to rest at the desired distance from the new 
and reaching to the outside of the arm without moving 55 planetary system. Or, we may strike button C and come 
his elbow. Let us put a head sensor on the traveller, but abruptly to full stop. 

suppress the position information from the sensor, using Abruptly stopping is a non-physical motion change; 
it only to calculate gazing direction from orientation. however, we are in virtual reality and we may suspend 
Now, further suppose that the traveller is moving virtu- the laws of physics in various ways. We double-click 
ally, for example in interstellar space. How is the travel- 60 button C and enter translation^ scan mode. The word 
ler to go to a star observed in his field of view? Once "scan" is discreetly displayed in the peripheral field of 
nearby and seeing an interesting planetary system to view as though part of a heads-up display. In scan 
explore, how is the traveller to orient himself to the mode, the pod responds to ball pushes by translating 
ecliptic of that planetary system for close-in navigation? with a speed, rather than an acceleration, proportional 
The reader will notice that a ground plane is sneaking 65 to ball displacement. We still get the motion coherence 
into the metaphor, but that is not a difficulty; the virtual cues of objects passing by in the peripheral field of 
space pod must also support effective navigation in view. However, we no longer need to overcome virtual 
spaces of dimensionality less than three. It is a fact of inertia; we are in motion only so long as we are pushing 
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the ball. Scan mode allows us quickly to move to an with rotations of the pod. FIG. 4 illustrates the two sets 

exact desired position. We double-click C again to exit of circles 22, 24 separated from one another by the 

scan mode. We strike button B to see the yellow pod displacement vector D reflecting the displacement of 

navigation circles overlayed with red circles, this time. the coordinate system X, Y, Z of the pod or sphere 20 

The red circles are aligned with the planetary coordi- 5 from the origin of the global, interstellar or galactic 

nate system. coordinate system X', Y', Z\ The pod coordinate sys- 

The foregoing scenario presents what seems a very tem X, Y, Z moves with a velocity vector V relative to 

natural navigation metaphor that could be easily mas- the global coordinate system. 

tered with a little practice. It implies the algorithms FIG. 4 illustrates how off-axis circles 25 may be su- 

necessary to map signals from the control ball to coordi- 10 perimposed on the great navigational circles 22 in the 

nate transformations. In normal mode, translational manner of degree tick marks. There are three orthogo- 

control signals map to translational accelerations. So nal sets of off-axis circles, one set for each of the three 

long as a translational input from the ball is present, a orthogonal great navigational circles 22X, 22Y, 22Z. 

proportional acceleration is delivered to the local pod When the pod and global great navigational circles are 

coordinate system with respect to the global space co- 15 superimposed together as in FIG. 3, then there are six 

ordinate system. Velocity accumulates independently in sets of off-axis circles available to the pilot. The pres- 

each coordinate direction. When the translational de- ence, intensity and transparency of the great naviga- 

grees of freedom of the ball are relaxed, motion contin- tional circles as well as of the off-axis circles 25 is con- 

ues according to the velocity that has accumulated. trolled by the user through a second (left) control ball 

Thus, translation is consistent with Newton's First Law 20 16' shown in FIG. 1. The user can select any one or all 

of Motion (a body in motion in a straight line continues 6f the six great navigational circles 22 and anyone or all 

in motion until acted upon by an external force). Trans- of the six sets of off-axis circles to be displayed or not 

lational motion changes only when the ball is pushed displayed at any time through the second control ball 

and the pod coordinate system is accelerated. 16'. The second control ball 16' also permits the user to 

Alternatively, one might imagine that the transla- 25 control the number of off-axis circles per degree of arc. 
tional degree of freedom of the ball control virtual The advantage is that as the pod rotates in response to 
rockets attached to the pod, in which case the accelera- commands from the traveller through the control ball 
tion follows from Newton's Second Law of Motion 16, the traveller can continue gazing at the surrounding 
F=ma with the mass, m, of the pod being a parameter scene while simultaneously feeling the magnitude and 
to the algorithm. In scan mode, ball input maps to trans- 30 direction of rotations of the pod. The pilot perceives 
lational velocity rather than acceleration. When the ball translational movement of the pod by the passage of 
is relaxed, motion stops, non-physically. Rotational scenery across his field of vision. FIG. 5 illustrates how 
(twist) control signals map to rotational velocities. So the two sets of circles appear through one narrow field 
long as a rotational input from the ball is present, a of view of the traveller in a particular orientation of the 
proportional velocity is delivered to the local pod coor- 35 pod reflecting random rotation about multiple axes. In 
dinate system with respect to the global space coordi- order to enhance the pilot's instantaneous feel of orien- 
nate system. When the rotational degrees of freedom of tation changes, each of the circles 22, 24 has period 
the ball are relaxed, rotational motion of the pod stops markings every so many degrees. For example, large 
and the acquired orientation obtains. One might imagine marks are placed every 45 degrees while smaller mark- 
that the rotational degrees of freedom of the ball control 40 ings axe placed every five degrees, 
virtual gimbal motors. The virtual gimbals are anchored FIG. 6 illustrates the architecture of the computer 19 
to the global coordinate system by virtual gyroscopes. and its interfaces with the control ball 16, the pilot head 
One might easily imagine that a more physical spin rotation sensor 17 and the pilot display 18. 
mode, in which rotational degrees of freedom are A memory 26 stores a model of the environment 
mapped to rotational accelerations, might be useful. 45 (global, oceanic space, interplanetary or interstellar) 
Switching among the modes should be under user con- through which the pod is to travel. Such a model com- 
trol. Software provides for all modes and maintains prises information defining relative locations of objects 
consistency across mode switches. (e.g., galaxies, stars and planets), the surface images of 

FIG. 3 illustrates the traveller's perception of the two those objects, their relative velocities in the galactic 

sets of circles superimposed on the observed scene. The 50 coordinate system, etc. Such informational models are 

traveller 10 is apparently surrounded by a sphere 20 on of the type that have been generated by computers 

which is fixed three co-spherical yellow orthogonal using images (radar and photographic) from mterplane- 

circles. The sphere 20 corresponds to the traveller's tary vehicles such as Galileo and Voyager, 

space pod centered on the three orthogonal axes X, Y, A pod translation/position processor 28 employs 

Z of the space pod and the three yellow circles 22x, 22y, 55 conventional dynamic algorithms to continuously com- 

22z have as their axes the three orthogonal axes of the pute and update the position and velocity of the pod 

space pod. The three orthogonal blue circles 24*', 24/, from pod acceleration commands received from the 

24/, also co-spherical with the sphere 20, have as their control ball 16 (or any equivalent six-degree-of-freedom 

axes the three orthogonal axes X', Y', Z' of a global or joy stick). 

interstellar coordinate system defined prior to "take 60 A pod rotation processor 30 employs conventional 

off". The global circles are parallel to the global coordi- dynamic algorithms to continuously compute and up- 

nate system. Both sets of circles are centered at the date the rotational orientation and rotational velocity of 

center of the sphere 20, the blue circles of course being the pod from pod rotational acceleration commands 

translated from the true origin of the global coordinate received from the control ball 16. 

system by translational movement of the pod and travel- 65 A pilot head rotation processor 32 employs conven- 

ler 10 to keep those circles concentric with the sphere tional dynamic algorithms to continuously compute and 

20. The traveller 10 perceives that the yellow circles are update the rotational orientation of the pilot's head 

stationary, while the blue circles appear move about relative to the pod coordinate system from rotational 
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position or velocity data received from the pilot head 
rotation sensor 17. A vector defining the pod orienta- 
tion in the galactic coordinate system is computed by 
the pod rotation processor 30 and transmitted to the 
pilot head rotation processor. The pilot head rotation 5 
processor modifies this vector to produce a vector de- 
fining the pilot head orientation in galactic coordinates. 
This vector defines the direction of the pilot's perspec- 
tive view of galactic space, and is transmitted by the 
pilot head rotation processor to a perspective view 10 
generating processor 34. The operation of the two rota- 
tion processors 30, 32 may be thought of as two succes- 
sive rotation matrix multiplications or transformations 
of a vector defining the perspective view direction, 
namely a first rotation in accordance with the pod's IS 
current orientation and a second rotation in accordance 
with the pilot's head orientation. This is necessary be- 
cause, even if the pod does not rotate, the image pres- 
ented to the pilot must change to reflect rotations of the 
pilot's head. Another conventional approach is to use 20 
homogeneous coordinates, combining translation and 
rotation in one 4-by-4 matrix product. 

From the galactic position of the pod computed by 
the pod translation/position processor 28, and the pilot 
head orientation relative to galactic space computed by 25 
the pod rotation and pilot head rotation processors 30, 
32, the perspective view generating processor 34 com- 
putes, from the model 26 of galactic space, the pilot's 
perspective view of galactic space, using, for example, 
image processing software demonstrated at the Jet Pro- 30 
pulsion Laboratories in Pasadena, Calif. Perspective 
computations are available in graphics processor hard- 
ware or can be quickly done in software. In this per- 
spective view, of course, objects closest to the pod 
appear relatively large while objects farther away ap- 35 
pear smaller, and objects approaching infinity appear to 
converge along lines of perspective, much like the pro- 
verbial railroad tracks on a flat landscape. 

The perspective view generating processor 34 trans- 
mits the pilot's perspective view of space to multi-image 40 
superpositioning electronics 35. 

In order to generate the image of the three orthogo- 
nal circles 22x, 22j>, 22z fixed to the pod coordinate 
system, a memory 36 stores the image viewed from the 
center of the sphere 20 of the three circles 22 apparently 45 
fixed to the virtual sphere 20 surrounding the pilot 10. 
The pilot head rotation processor 32 rotates the image 
of the circles 22 so as to present the correct portion 
thereof to the pilot's relatively narrow field of view, 
depending upon the current orientation of the pilot's 50 
head. This produces a head-rotated image of the pod 
great circles, which is transmitted to the multi-image 
superpositioning electronics 35. 

In order to generate the image of the three orthogo- 
nal circles 24 oriented with the galactic coordinate 55 
system (but co-spherical on the sphere 20), a memory 38 
stores the image of the three orthogonal circles 24 
viewed from the center of the sphere 20. Since the 
circles 24 are of a fixed orientation relative to the galac- 
tic coordinate system, their image must move with all 60 
rotations of the pod or sphere 20. Therefore, the image 
stored in the memory 38 is transmitted to the pod rota- 
tion processor 30 for rotation thereof in accordance 
with rotations of the pod. The resulting pod-rotated 
image of the three orthogonal circles is transmitted to 65 
the pilot head rotation processor 32 for rotation with 
the pilot's head. The resulting rotated image is of a 
portion of the three circles 24 encompassed by the pi- 
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lofs relatively narrow field of view and corresponding 
to the pilot head current orientation in galactic coordi- 
nates. This final rotated image of the three circles 24 is 
transmitted to the multi-image superpositioning elec- 
tronics 35. 

The superpositioning electronics 35, which may be an 
image processor, assigns a yellow color to the image of 
the three circles 22 and a blue color to the image of the 
three circles 24 and superimposes these images onto the 
image of the pilot's perspective view of galactic space 
to generate a superimposed image of the pilot's perspec- 
tive view through the transparent sphere 20 defined by 
six great circles co-spherical with the sphere 20, namely 
the three yellow circles 22x, 22y, 22z and the three blue 
circles 24x' f 24j/, 24/. This final image is transmitted to 
the pilot binocular, display 18. 

Using well-known techniques, the superpositioning 
electronics ; may generate two versions of the image 
stereoscopically displayed to the pilot's eyes by respec- 
tive right and left eye displays in the display 18, in ac- 
cordance with a true stereoscopic image. Preferably, 
the yellow and blue navigation circles are transparent 
images which do not obscure any objects in the pilot's 
field of view. For this purpose, the superpositioning 
electronics 35 employs alpha-blending in accordance 
with well-known techniques to compute the transparent 
images of the yellow and blue navigational circles. The 
superpositioning electronics 35 also responds to com- 
mands from the second control ball 16' to add or delete 
or vary the intensity or transparency of the off-axis 
circles 25 of any one or all of the two sets of orthogonal 
great navigational circles 22 and to change the number 
of off-axis circles per degree of arc, or to shorten the 
off-axis circles 25 to small degree tick marks, using 
appropriate conventional image display techniques. 

In order to realistically follow pilot head movements 
and pod movements, the image presented by the pilot 
display 18 must be periodically updated very rapidly. 
This requires that the entire system compute a new 
image for display rapidly in accordance with the latest 
changes in commanded translational and rotational ac- 
celerations from the control ball 16 and in accordance 
with changes in pilot head orientation sensed by the 
pilot head sensor 17. Thus, each binocular display in the 
pilot display 18 receives a new "video frame" at a very 
rapid rate, preferably greater than the convention tele- 
vision frame rate of 30 frames per second. For this pur- 
pose, the processors in the system of FIG. 6 are of the 
highest power and speed. 

How the system of FIG. 6 operates is illustrated in 
the flow diagram of FIG. 7. The first step (block 40 of 
FIG. 7) is to initialize or define the location and orienta- 
tion of the pod coordinate system X, Y, Z relative to the 
galactic coordinate system X', Y', Z'. Next, the pod 
translation/position processor 28 integrates pod transla- 
tional accelerations commanded by the control ball 16 
to compute a new position of the pod or sphere 20 
(block 42 of FIG. 7). This position is transformed to 
galactic coordinates (block 44 of FIG. 7). The pod 
rotation processor 30 integrates pod rotational accelera- 
tions commanded by the control ball 16 to compute a 
new pod orientation (block 46 of FIG. 7). This orienta- 
tion is transformed to galactic coordinates (block 48 of 
FIG. 7). The pilot head rotation processor 32 integrates 
changes in the pilot head orientation sensed by the pilot 
head rotation sensor 17 and computes a new orientation 
of the pilot's head in pod coordinates (block 50 of FIG. 
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7). This new pilot head orientation is transformed to figures whose relative orientations provide the user an 

galactic coordinates (block 52 of FIG. 7). instantaneous reference of orientation with respect to a 

The perspective view generating processor 34 then fixed coordinate system, 

generates the pilot's perspective view of galactic space 3. The apparatus of claim 2 wherein said set of geo- 

from the model 26 based upon the pod's current position 5 metric figures apparently surround the user, 

(block 54), any changes in the pod*s orientation (block 4. The apparatus of claim 3 wherein the set of geo- 

56) and any changes in the orientation of the pilot's head metric figures superimposed on the scene and appar- 

(block 58 of FIG. 7). ently surrounding the user include a first set of geomet- 

Frior to initialization, the memory 38 stores data ric figures whose orientations are fixed to the user's 

defining the image of the three orthogonal circles 22 10 vehicle and a second set of geometric figures whose 

fixed to the pod coordinate system and centered at the orientations are fixed with respect to a reference coordi- 

pod coordinate axis origin at the center of the virtual nate system. 

sphere 20 (block 60 of FIG. 7). For this purpose, image 5. The apparatus of claim 4 wherein the first set of 

pixels in the memory 38 are best defined in spherical geometric figures comprise a first set of orthogonal 

coordinates. Prior to initialization, the memory 36 15 great circles about three orthogonal axes of the flight 

stores data defining the image of the three orthogonal vehicle and wherein the second set of geometric figures 

circles 24 fixed to the galactic coordinate system but comprises a second set of orthogonal great circles about 

centered at the origin of the pod coordinate system at the three orthogonal axes of said reference coordinate 

the center of the sphere 20 (block 62 of FIG. 7), For this system. 

purpose, the image pixels in the memory 36 are best 20 6. The apparatus of claim 5 wherein all of said circles 

defined in spherical coordinates. are concentric with a virtual sphere corresponding to 

After the step of block 48, the pod rotation processor said vehicle and centered at the user's head. 

30 rotates the image from the memory 38 of the three 7. The apparatus of claim 6 wherein said images of 

orthogonal circles 24 fixed to the galactic coordinate said circles further comprise markings at periodic arc 

system in accordance with the latest rotation of the pod 25 lengths along each of the circles which permit the user 

relative to galactic coordinates (block 64 of FIG. 7). to instantly gauge approximate angles of rotation about 

The pilot head rotation processor 32 then rotates the any axis, said marking being one of: (a) degree tick 

pod-rotated image of the circles 24 (block 66 of FIG. 7) marks, or (b) off-axis circles distributed along the axes 

in accordance with the current pilot head orientation. of each coordinate system at periodic intervals. 

Also, the pilot head rotation processor 32 rotates the 30 8. The apparatus of claim 1 further comprising means 

image of the circles 22 in accordance with the current for permitting the user to scan among a set of simulated 

pilot head orientation (block 68 of FIG. 7). The rotated velocity images for easy navigation in simulated flight 

images of the circles 22, 24 are superimposed onto the of a non-physical changing motion training exercise, 

pilot perspective view of the scene by the display elec- 9. A flight control apparatus for displaying to a user 

tronics 35 (block 70 of FIG. 7) and the resulting image 35 the image of a scene surrounding a vehicle traveling in 

is displayed by the display 18 as the next video frame inertia! space comprising: 

(block 72 of FIG. 7). The system then restarts the entire means for imparting six-degree-of-freedom of motion 

process at the steps of blocks 42, 46 and 50 to produce control by said user; 

the next video frame. means for displaying to the user the image of a scene 

While the invention has been described in detail by 40 surrounding a vehicle containing the user traveling 

specific references to preferred embodiments, it is un- through inertia! space from commands transmitted 

derstood that variations and modifications thereof may by said means for imparting six-degree-of-freedom 

be made without departing from the true spirit and of motion control; and 

scope of the invention. means for superimposing on said image a superim- 

What is claimed is: 45 posed figure providing the user an instant reference 

1. An apparatus for displaying to a user the image of of rotational orientation wherein said image is free 
a scene surrounding a vehicle traveling in inertial space of a ground plane, thereby assisting said user pre- 
in response to rotational and translational acceleration forming the appropriate navigation in the inertial 
commands by the user in six degrees of freedom, the space. 

apparatus comprising: 50 10. The apparatus of claim 9 wherein said superim- 

a full-field-of-view display for displaying said image posed image comprises a superimposed set of geometric 

of the scene surrounding the vehicle to the user; figures whose relative orientations provide the user an 

means for producing images on said full-field-of-view instantaneous reference of orientation with respect to 

display, said images changing to correspond to a one of a plurality of coordinate systems. 

new perspective of the user in response to said 55 11. The apparatus of claim 10 wherein said set of 

rotational and translational acceleration commands geometric figures apparently surround the user. 

by the user in six-degrees-of-freedom; 12. The apparatus of claim 11 wherein the set of geo- 

means for the user to impart said rotational and trans- metric figures superimposed on the scene and appar- 

lational acceleration commands to said image pro- ently surrounding the user include a first set of geomet- 

ducing means; and 60 ric figures whose orientations are fixed to the user's 

means for superimposing on said image a superim- vehicle and a second set of geometric figures whose 

posed figure providing the user an instant reference orientations are fixed with respect to a reference coordi- 

of rotational orientation wherein said image is free nate system. 

of a ground plane, thereby assisting said user pre- 13. The apparatus of claim 12 wherein the first set of 

forming the appropriate navigation in the inertial 65 geometric figures comprise a first set of orthogonal 

space. great circles about three orthogonal axes of the flight 

2. The apparatus of claim 1 wherein said superim- vehicle and wherein the second set of geometric figures 
posed image comprises a superimposed set of geometric comprises a second set of orthogonal great circles about 
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the three orthogonal axes of said reference coordinate 
system. 

14. The apparatus of claim 13 wherein all of said 
circles are concentric with a virtual sphere correspond- 
ing to said vehicle and centered at the user's head. 5 

15. The apparatus of claim 14 wherein said images of 
said circles further comprise markings at periodic arc 
lengths along each of the circles which permit the user 
to instantly gauge approximate angles of rotation about 
any axis. 10 

16. The apparatus of claim 15 wherein said means for 
imparting six-degree-of-freedom motion control com- 
prises a control ball moveable in six degrees of freedom 
and connected to said means for displaying and a sec- 
ondary control ball connected to said means for display- IS 
ing for controlling parameters of said great circles in- 



cluding at least one of transparency, intensity, presence, 
and pitch of said markings along said great circles. 

17. The apparatus of claim 16 wherein said means for 
displaying comprises: 
means for storing a model defining said inertial space 
and locations and appearances of objects therein; 
means for computing translational and rotational 
positions of said user in said inertial space from 
commands transmitted by said means for imparting 
six-degree-of-freedom motion control, and 
means for producing from said translational and rota- 
tional positions and from said model an image of a 
perspective view of said inertial space as observed 
from a vantage point of said user. 
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